8 research outputs found
Predictive World Models from Real-World Partial Observations
Cognitive scientists believe adaptable intelligent agents like humans perform
reasoning through learned causal mental simulations of agents and environments.
The problem of learning such simulations is called predictive world modeling.
Recently, reinforcement learning (RL) agents leveraging world models have
achieved SOTA performance in game environments. However, understanding how to
apply the world modeling approach in complex real-world environments relevant
to mobile robots remains an open question. In this paper, we present a
framework for learning a probabilistic predictive world model for real-world
road environments. We implement the model using a hierarchical VAE (HVAE)
capable of predicting a diverse set of fully observed plausible worlds from
accumulated sensor observations. While prior HVAE methods require complete
states as ground truth for learning, we present a novel sequential training
method to allow HVAEs to learn to predict complete states from partially
observed states only. We experimentally demonstrate accurate spatial structure
prediction of deterministic regions achieving 96.21 IoU, and close the gap to
perfect prediction by 62% for stochastic regions using the best prediction. By
extending HVAEs to cases where complete ground truth states do not exist, we
facilitate continual learning of spatial prediction as a step towards realizing
explainable and comprehensive predictive world models for real-world mobile
robotics applications. Code is available at
https://github.com/robin-karlsson0/predictive-world-models.Comment: Accepted for IEEE MOST 202
ViCE: Visual Concept Embedding Discovery and Superpixelization
Recent self-supervised computer vision methods have demonstrated equal or
better performance to supervised methods, opening for AI systems to learn
visual representations from practically unlimited data. However, these methods
are classification-based and thus ineffective for learning dense feature maps
required for unsupervised semantic segmentation. This work presents a method to
effectively learn dense semantically rich visual concept embeddings applicable
to high-resolution images. We introduce superpixelization as a means to
decompose images into a small set of visually coherent regions, allowing
efficient learning of dense semantics by swapped prediction. The expressiveness
of our dense embeddings is demonstrated by significantly improving the SOTA
representation quality benchmarks on COCO (+16.27 mIoU) and Cityscapes (+19.24
mIoU) for both low- and high-resolution images
Learning to Predict Navigational Patterns from Partial Observations
Human beings cooperatively navigate rule-constrained environments by adhering
to mutually known navigational patterns, which may be represented as
directional pathways or road lanes. Inferring these navigational patterns from
incompletely observed environments is required for intelligent mobile robots
operating in unmapped locations. However, algorithmically defining these
navigational patterns is nontrivial. This paper presents the first
self-supervised learning (SSL) method for learning to infer navigational
patterns in real-world environments from partial observations only. We explain
how geometric data augmentation, predictive world modeling, and an
information-theoretic regularizer enables our model to predict an unbiased
local directional soft lane probability (DSLP) field in the limit of infinite
data. We demonstrate how to infer global navigational patterns by fitting a
maximum likelihood graph to the DSLP field. Experiments show that our SSL model
outperforms two SOTA supervised lane graph prediction models on the nuScenes
dataset. We propose our SSL method as a scalable and interpretable continual
learning paradigm for navigation by perception. Code released upon publication.Comment: Under revie
Long-term activation of anti-tumor immunity in pancreatic cancer by a p53-expressing telomerase-specific oncolytic adenovirus
Background: Pancreatic cancer is an aggressive, immunologically “cold” tumor. Oncolytic virotherapy is a promising treatment to overcome this problem. We developed a telomerase-specific oncolytic adenovirus armed with p53 gene (OBP-702).
Methods: We investigated the efficacy of OBP-702 for pancreatic cancer, focusing on its long-term effects via long-lived memory CD8 + T cells including tissue-resident memory T cells (TRMs) and effector memory T cells (TEMs) differentiated from effector memory precursor cells (TEMps).
Results: First, in vitro, OBP-702 significantly induced adenosine triphosphate (ATP), which is important for memory T cell establishment. Next, in vivo, OBP-702 local treatment to murine pancreatic PAN02 tumors increased TEMps via ATP induction from tumors and IL-15Rα induction from macrophages, leading to TRM and TEM induction. Activation of these memory T cells by OBP-702 was also maintained in combination with gemcitabine+nab-paclitaxel (GN) in a PAN02 bilateral tumor model, and GN + OBP-702 showed significant anti-tumor effects and increased TRMs in OBP-702-uninjected tumors. Finally, in a neoadjuvant model, in which PAN02 cells were re-inoculated after resection of treated-PAN02 tumors, GN + OBP-702 provided long-term anti-tumor effects even after tumor resection.
Conclusion: OBP-702 can be a long-term immunostimulant with sustained anti-tumor effects on immunologically cold pancreatic cancer
LiDAR Point Cloud Augmentation for Adverse Conditions Using Conditional Generative Model
The perception systems of autonomous vehicles face significant challenges under adverse conditions, with issues such as obscured objects and false detections due to environmental noise. Traditional approaches, which typically focus on noise removal, often fall short in such scenarios. Addressing the lack of diverse adverse weather data in existing automotive datasets, we propose a novel data augmentation method that integrates realistically simulated adverse weather effects into clear condition datasets. This method not only addresses the scarcity of data but also effectively bridges domain gaps between different driving environments. Our approach centers on a conditional generative model that uses segmentation maps as a guiding mechanism to ensure the authentic generation of adverse effects, which greatly enhances the robustness of perception and object detection systems in autonomous vehicles, operating under varied and challenging conditions. Besides the capability of accurately and naturally recreating over 90% of the adverse effects, we demonstrate that this model significantly improves the performance and accuracy of deep learning algorithms for autonomous driving, particularly in adverse weather scenarios. In the experiments employing our augmented approach, we achieved a 2.46% raise in the 3D average precision, a marked enhancement in detection accuracy and system reliability, substantiating the model’s efficacy with quantifiable improvements in 3D object detection compared to models without augmentation. This work not only serves as an enhancement of autonomous vehicle perception systems under adverse conditions but also marked an advancement in deep learning models in adverse condition research
Machine learning algorithm‐based estimation model for the severity of depression assessed using Montgomery‐Asberg depression rating scale
Abstract Aim Depressive disorder is often evaluated using established rating scales. However, consistent data collection with these scales requires trained professionals. In the present study, the “rater & estimation‐system” reliability was assessed between consensus evaluation by trained psychiatrists and the estimation by 2 models of the AI‐MADRS (Montgomery‐Asberg Depression Rating Scale) estimation system, a machine learning algorithm‐based model developed to assess the severity of depression. Methods During interviews with trained psychiatrists and the AI‐MADRS estimation system, patients responded orally to machine‐generated voice prompts from the AI‐MADRS structured interview questions. The severity scores estimated from two models of the AI‐MADRS estimation system, the max estimation model and the average estimation model, were compared with those by trained psychiatrists. Results A total of 51 evaluation interviews conducted on 30 patients were analyzed. Pearson's correlation coefficient with the scores evaluated by trained psychiatrists was 0.76 (95% confidence interval 0.62–0.86) for the max estimation model, and 0.86 (0.76–0.92) for the average estimation model. The ANOVA ICC rater & estimation‐system reliability with the evaluation scores by trained psychiatrists was 0.51 (−0.09 to 0.79) for the max estimation model, and 0.75 (0.55–0.86) for the average estimation model. Conclusion The average estimation model of AI‐MADRS demonstrated substantially acceptable rater & estimation‐system reliability with trained psychiatrists. Accumulating a broader training dataset and the refinement of AI‐MADRS interviews are expected to improve the performance of AI‐MADRS. Our findings suggest that AI technologies can significantly modernize and potentially revolutionize the realm of depression assessments